The present invention relates to an image sensing apparatus capable of determining an optimum image sensing conditions for obtaining a three-dimensional image. Further, the present invention also relates to an image processing apparatus which edits a three-dimensional image in accordance with the image sensing conditions.
Conventionally, there is a technique, such as the one published on the Journal of Television Society, Vol. 45, No. 4 (1991), pp. 453–460, to obtain a three-dimensional shape of an object. There are basically two methods, a passive method and an active method, for obtaining the three-dimensional shape of the object, as described in the above article.
A typical method as the passive method is a stereo imaging method which performs triangulation on an object by using two cameras. In this method, corresponding points of a part of an object are searched in both the right and left images, and the position of the object in the three dimensional space is measured on the basis of the difference between the positions of the searched corresponding points in the right and left images.
Further, there are a method using a range finder and a slit projection method as the typical active method. In the former method, distance to the object is obtained by measuring time since emitting light toward the object until receiving the light reflected by the object. In the latter method, a three-dimensional shape is measured on the basis of deformation of a shape of a light pattern, whose original pattern is a slit shape, projected on an object.
However, the main purpose of the aforesaid stereo imaging method is to calculate information on distance between fixed positions where the cameras are set and the object, and not to measure the entire object. Consequently, a three-dimensional shape can not be obtained in high precision.
Further, an apparatus adopting the active method is large, since it has to emit a laser beam, for example, to an object, thereby manufacturing cost is high.
Generation of a three-dimensional shape of the object on the basis of two-dimensional images requires a plurality of images sensed at a plurality of image sensing points. However, since the object has a three-dimensional shape, image sensing parameters (e.g., depth of focus, angle of view) which are suited to the object and each image sensing point needs to be set for performing image sensing from a plurality of image sensing points.
However, in any of the aforesaid methods, cameras are not controlled flexibly enough to respond to a dynamic image sensing method, such as sensing an object while moving around it.
Therefore, the present invention is aimed to solve the aforesaid problem, i.e., to realize a dynamic image sensing method in which an object is sensed at a plurality of image sensing points around it.
Meanwhile, an image of an object which is seen from an arbitrary viewpoint is sometimes reproduced on a two-dimensional display on the basis of obtained three-dimensional data of the object.
For example, it is possible to input images sensed by an electronic camera into a personal computer, or the like, and edit them. In this case, a scene is divided into a plurality of partial scenes, and then is sensed with an electronic camera. The images corresponding to the plurality of partial scenes is projected with having some overlapping portions. More specifically, the sensed images are inputted into a personal computer, then put together by using an application software so that the overlapping portions are projected overlapped each other. Thereby, it is possible to obtain an image of far wider angle of view than that of an image obtained in a single image sensing operation by the electronic camera.
However, the main purpose of the aforesaid stereo imaging method is to calculate information on distance between a fixed position where the camera is set and the object, and not to measure the three-dimensional shape of the entire object.
Further, since a laser beam is emitted to the object in the active method, it is troublesome to use an apparatus adopting the active method. Furthermore, in any conventional method, cameras are not controlled flexibly enough to respond to a dynamic image sensing method, such as sensing an object while moving around it.
In addition, two view finders are necessary in the conventional passive method using two cameras, and it is also necessary to perform image sensing operation as seeing to compare images on the two view finders, which increases manufacturing cost and provides bad operability. For instance, there are problems in which it takes time to perform framing or it becomes impossible to obtain a three-dimensional shape because of too small overlapping area.
Further, an image generally dealt with in an office is often printed out on paper eventually, and types of images to be used may be a natural image and a wire image which represents an object with outlines only. In the conventional methods, however, to display an image of an object faithfully on a two-dimensional display on the basis of obtained three-dimensional shape data of the object is the main interest, thus those methods are not used in offices.